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Abstract 

We give necessary and sufficient conditions for the (bounded) law 
of the iterated logarithm for [/-statistics in Hilbert spaces. As a tool 
we also develop moment and tail estimates for canonical Hilbert-space 
valued [/-statistics of arbitrary order, which are of independent inter- 
est. 
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1 Introduction 

In the last two decades we have witnessed a rapid development in the asymp- 
totic theory of [/-statistics, boosted by the introduction of the so called 
'decoupling' techniques (see 0EJE]), which allow to treat [/-statistics con- 
ditionally as sums of independent random variables. This approach yielded 
better understanding of [/-statistics versions of the classical limit theorems 
of probability. Necessary and sufficient conditions were found for the strong 
law of large numbers [T7], the central limit theorem [19\ [10] and the law 
of the iterated logarithm [2] . Also some sharp exponential inequalities 
for canonical [/-statistics have been found 00] 03]. Analysis of the afore- 
mentioned results shows an interesting phenomenon. Namely, the natural 
counterparts of the necessary and sufficient conditions for sums of i.i.d. ran- 
dom variables ([/-statistics of degree 1), remain sufficient for [/-statistics 
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of arbitrary degree, but with an exception for the CLT, they cease to be 
necessary. The correct conditions turn out to be much more involved and 
are expressed for instance in terms of convergence of some series (LLN) or 
as growth conditions for some functions (LIL). 

A natural problem is an extension of the above results to the infinite- 
dimensional setting. There has been some progress in this direction, and 
partial answers have been found, usually under the assumption on the geo- 
metrical structure of the space in which the values of a [/-statistic are taken. 
In general however the picture is far from being complete and the necessary 
and sufficient conditions are known only in the case of the CLT for Hilbert 
space valued U -statistics (see [5j [10] for the proof of sufficiency in type 2 
spaces and necessity in cotype 2 spaces respectively). 

In this article we generalize to separable Hilbert spaces the results from 
[2] on necessary and sufficient conditions for the LIL for real valued U- 
statistics. The conditions are expressed only in terms of the [/-statistic 
kernel and the distribution of the underlying i.i.d. sequence and can be also 
considered a generalization of results from [13] . where the LIL for i.i.d. sums 
in Hilbert spaces was characterized. We consider only the bounded version 
of the LIL and do not give the exact value of the lim sup nor determine the 
limiting set. Except for the classical case of sums of i.i.d. random variables, 
the problem of finding the lim sup is at the moment open even in the one 
dimensional case (see [31 02 [15] for some partial results) and the problem of 
the geometry of the limiting set and the compact LIL is solved only under 
suboptimal integrability conditions [3]. 

The organization of the paper is as follows. First, in Section [3] we prove 
sharp exponential inequalities for canonical [/-statistics, which generalize 
the results of [TJ [8] for the real-valued case. Then, after recalling some 
basic facts about the LIL we give necessary and sufficient condition for the 
LIL for decoupled, canonical [/-statistics (Theorem [2]) . The quite involved 
proof is given in the two subsequent sections. Finally we conclude with 
our main result (Theorem [J]) , which gives a characterization of the LIL for 
undecoupled U -statistics and follows quite easily from Theorem [2] and the 
one dimensional result. 

2 Notation 

For an integer d, let (Xi)i e jq, )ieN,i<k<d be independent random vari- 
ables with values in a Polish space S, equipped with the Borel u-field T. 
Let also (£i)ieN> {^)i&i,\<k<d be independent Rademacher variables, in- 
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dependent of (Xj) ieN , (X^) ieN:1 < k < d . 

Consider moreover measurable functions h\: S d —> H, where (H, \ • |) is a 
separable Hilbert space (we will denote both the norm in H and the absolute 
value of a real number by | • | , the context will however prevent ambiguity) . 

To shorten the notation, we will use the following convention. For i = 
(h,...,id) e {!,..., n} d we will write Xi (resp. Xf cc ) for (X h X id ), 



(resp. (X^\ . . . , X\^')) and e\ (resp. ef cc ) for the product 



-(d) 



'id 



(resp. 



.(<*)> 



£l"'), the notation being thus slightly inconsistent, which however 



should not lead to a misunderstanding. The [/-statistics will therefore be 
denoted 



5>(Xi) 

i£l d 

£ ^( x i cc ) 

|i|<n 

£ ef cc /ii(Xf cc ) 

|i|<n 



(an undecoupled [/-statistic) 



(a decoupled U -statistic) 



(an undecoupled randomized [/-statistic) 



(a decoupled randomized [/-statistic) , 



where 



In = {i: |i| < + i k for j / fc}. 
Since in this notation {1, . . . , d} = l\ we will write 

J d = {l,2,...,d}. 

Throughout the article we will write L^, L to denote constants depending 
only on d and universal constants respectively. In all those cases the values 
of a constant may differ at each occurrence. 

For / C I d , we will write Ej to denote integration with respect to vari- 
ables (X^)i e jqj e i. We will consider mainly canonical (or completely de- 
generated) kernels, i.e. kernels h\, such that for all j G Id, Ej/ii(Xf cc ) = 
a.s. 
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3 Moment inequalities for [/-statistics in Hilbert 
space 

In this section we will present sharp moment and tail inequalities for Hilbert 
space valued [/-statistics, which in the sequel will constitute an important 
ingredient in the analysis of the LIL. These estimates are a natural general- 
ization of inequalities for real valued U-statistics presented in pQ. 
Let us first introduce some definitions. 

Definition 1. For a nonempty, finite set I let Vi be the family consisting of 
all partitions J = {J%, . . . , of I into nonempty, pairwise disjoint subsets. 
Let us also define for J as above deg(J') = k. Additionally let V% = {0} 
with deg(0) = 0. 

Definition 2. For a nonempty set I C 1^ consider J = {J\, . . . , J^} E Vj. 

For an array (h[) i£l d of H-valued kernels and fixed value ofijc, define 

dcg(J) 

11(^11.7 = sup { | ^[^(Xf-) [J /4 } (X5 C )]|:/^:S J ^R 

ij 3=1 

E E l^( X i?)| 2 < 1 for i = 1, . . . ,deg(J)}. 
Let moreover II (/ii)i ffl L = \hi\. 



Remark It is worth mentioning that for I = 1^, [| • \\j is a deterministic 
norm, whereas for I C 1^ it is a random variable, depending on X^° c . 

Quantities given by the above definition suffice to obtain precise moment 
estimates for real valued [/-statistics. However, to bound the moments of 
[/-statistics with values in general Hilbert spaces, we will need to introduce 
one more definition. 

Definition 3. For nonempty sets K C I C L^ consider J = {J\, . . . , </&} E 
Vi\k- For an array (h{) ieI d of H-valued kernels and fixed value ofijc, define 

deg(J) 



MkhUj = sup{|^E / [(/ li (Xf cc ), 5iK (Xf-)) J] /£J(Xgp]|: 

ij 3=1 
fg> : - R, g iK : Y, K —* H , E £ \g- lR (Xf-)| 2 < 1 

IK 

E E \fh.( X ijP\ 2 < 1 for J = 1, • • .,deg(J)}. 
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Remark One can see that the only difference between the above definition 
and Definition [2] is that the latter distinguishes one set of coordinates and 
allows functions corresponding to this set to take values in H. Moreover, 
since the norm in H satisfies | • | = sup|0| <1 ((/>, •), we can treat Definition [2] 
as a counterpart of Definition for K = 0. We will use this convention to 
simplify the statements of the subsequent theorems. Thus, from now on, we 
will write 

II • U,j '■= II • 

Example For d = 2 and / = {1,2}, the above definition gives 
\\(hij(Xi, 5j))i,jll0,{{i,2}} = sup hij{Xi, Yj \J, i iX;.Y i 



= sup /EV^/^pQ,^)} 2 , 
<PeH,\^\<i y y 

\\(hij(Xi, Yj))ij ||0 i{ {i}{ 2 }} = sup | |e^ hij(Xi, Yj)fi(Xi)gj{Yj 

ij 

^E/(x,) 2 ,^E 5 (y,) 2 <i}, 
||(^(x i ,y,0)M'll{i},{{2}} = sup{E^(/ l (x i ),^(x i ,y J 0)*(^): 

ij 

E^l/^)! 2 ^^^) 2 ^!}, 

» j 

\\(h l3 (Xi, y j )) M -|| {li2} ,0 = sup {Ej^ifojiXi, Yj), hij{Xi,Y-)) : 

ij 

E^lfiX^Kl) 



'j 



We can now present the main result of this section. 
Theorem 1. For any array of H -valued, completely degenerate kernels 
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and any p > 2, we have 



E|£MX? ec )r<^( £ E P p{ * IC+desJ/2) ^ max ||(^ 11^)- 
i K<zici d jeVi\ K 1/c 

The proof of the above theorem proceeds along the lines of arguments 
presented in [TJ [8] . In particular we will need the following moment estimates 
for suprema of empirical processes [8] . 

Lemma 1 ([HJ Proposition 3.1], see also [U Theorem 12]). Let X%, . . . ,X n 

be independent random variables with values in (E, J-) and T be a countable 
class of measurable real functions on X, such that for all f £ T and i £ 
I n , Kf(Xi) = and E/(Xj) 2 < oo. Consider the random variable S : = 
sup /gr | Yji f( x i)\- Then for all p > 1, 



ES P < L p 



{ES) P + f /2 a p + fE max sup \ f(Xi)\ p 

1 feT 



where 



a 2 = supVE/(X i ) 2 . 

ft* i 



We will also need the following technical lemma. 

Lemma 2 (Lemma 5 in [I]). For a > and arbitrary nonnegative kernels 
g\: S d — » M + and p > 1 uie /iai>e 



p ap Emax 9 f + £ p #/p E / max(^E / c 5i ) 



Before stating the next lemma, let us introduce some more definitions, 
concerning ^7-norms of deterministic matrices 

Definition 4. Let {a\) l&I d be a d-indexed array of real numbers. For J = 
{Ji, . . . , J fc } £ Vi d define 

\\{a-M\j = sup { 5>*« ..•*(«: J>«) 2 <!,..., J>«) 2 < l}. 



We will also need 
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Definition 5. For i G N rf_1 x I n let a\: E — > M &e measurable functions 
and Z\,...,Z n be independent random variables with values in S. For a 
partition J = { Ji, . . . , J^} € "P/ d (ci G J]J, Zei us define 



( a i( z i d ))i\\j = sup{ 



E(^) 2 si--E(<) 2 <i}. 



*J 2 

Remark All the definitions of norms presented so far, seem quite similar 
and indeed they can be all interpreted as injective tensor-product norms on 
proper spaces. We have decided to introduce them separately by explicit 
formulas, because this form appears in our applications. 

The next lemma is crucial for obtaining moment inequalities for canon- 
ical real- valued [/-statistics of order greater than 2. In the context of U- 
statistics in Hilbert spaces we will need it already for d = 2. 

Lemma 3 (Theorem 5 in pQ). Let Z\, . . . , Z n be independent random vari- 
ables with values in (£, J 7 ). For i G N d_1 x I n let a\: £ —* R be measurable 
functions, such that Kzai(Zi d ) = 0. Then, for all p > 2 we have 

E\\C£ai(Z id )) ild J\<L d £ p^^^Ua^Mj 
i d Jev Id 



+ L d £ p 1 +( 1 +^W- d )/ 2 ./Emax||(a i (^)) i/d _J|^, 



where \\-\\ denotes the norm of a (d-l)-indexed matrix, regarded as a (d— 1)- 
linear operator on (h)^ 1 (thus the \\ ■ \\{i}...{d-i}~ norm * n our notation). 

To prove Theorem [TJ we will need to adapt the above lemma to be able 
to bound the (K, j7)-norms of sums of independent kernels. 

Definition 6. We define a partial order -< on Pi as 

if and only if for all I El, there exists J G J , such that I C J. 



7 



Lemma 4. Assume that E|/ij(X^ cc )| 2 < oo. Then for any K C i^-i and 
J" = {Ji, . . . , J fe } € Vi^K and al1 P> 2 , 

E4CEk(xt ec )K d _jK,j (i) 

<lJ y, p^^^mKhjc 

\ KCLCI d , >CeV Id \ L : 

JU{K,{d}}^KU{L} 



+ £ p l HdeS K- d e S J)/2 Edm ^ mhd 

KQLCi d _ 1: icer Id _ AL : V 
JU{K}<KU{L} 

Remark In the above lemma we slightly abuse the notation, by identifying 
for if = the partition {0} U J with J. 



Given Lemma El the proof of Lemma H] is not complicated, the main idea 
is just a change of basis, however due to complicated notation it is quite 
difficult to write it directly. We find it more convenient to write the proof 
in terms of tensor products of Hilbert spaces. 
Let us begin with a classical fact. 

Lemma 5. Let H be a separable Hilber space and X a T,-valued random 
variable. Then H <g> L 2 (X) ~ L 2 (X,H), where L 2 (X,H) is the space of 
square integrable random variables of the form f(X), f : S — > H -measurable. 
With the above identification, for h £ H , f(X) £ L 2 (X), we have h®f(X) = 
hf(X)£L 2 (X,H). 

Proof of Lemma^ To avoid problems with notation, which would lengthen 
an intuitively easy proof, we will omit some technical details, related to 
obvious identification of some tensor product of Hilbert spaces (in the spirit 
of Lemma [5]). Similarly, when considering linear functionals on a space, 
which can be written as a tensor product in several ways, we will switch to 
the most convenient notation, without further explanations. 
Let 

H = H®[® leK (®UL\XP)} ~ © M < n L 2 (X?-,#) 
and, for j = 1, ... , k, 

Hi = ® /eJj (©r=i£ 2 (^ (/) )) ~ ©ii^tfcxgp. 



S 



In the case K = 0, we have (using the common convention for empty 
products) Hq ~ H. 

For id = 1, . . . , n and fixed value of X^ , let Aj d be a linear functional on 
H = ®\ ild _ 1 \< n L 2 (X.f^_ i ,H) ~ ^ =0 F fc , d given by (^(X?**))^^ G 
with the formula 

(x£ x )) ild _ 1 ) = (0» Id _ 1 (xf- 1 , (^(xf-))^, 

= £ %,..., d -i}(^ d _ 1 (xf;;_ i ),/ ii (xf cc )) / ,. 

As functions of Aj d = are independent random linear func- 

tional. Thus they determine also random (k + l)-linear functionals on 
®j=o H k, given by 

(h , hi,..., h k ) i->- A id (h 8> /ii (8) . . . (g> 

If we denote by || • || the norm of a + l)-linear functional, the left hand-side 
of (pQ), can be written as 

n 

Moreover, denoting by \\Ai d \\hs the norm of Ai d seen as a linear operator on 
®j =0 Hj (by analogy with the Hilbert-Schmidt norm of a matrix), we have 

n 

Y J nA t A^)\\h = \\(h& l \\j d ,,<^, 

so the sequence Ai d [X.\ ) , determines a linear functional A on H® [©f d=1 L 2 (xj )] ~ 
©| i |< n L 2 (Xf cc , J H') ~ ®f d=1 L 2 (X^ ] ,H), given by the formula 

n 

A( 91 (X[\ . . .,g n (X^)) = E E[A d (X^)(9iA^f))}. 

»d=i 

It is easily seen, that if we interpret the domain of this functional as ©|j|< n L 2 (X^ ec , H), 
then it corresponds to the multimatrix (/ii(X^ ec ))i. 

Let us now introduce the following notation, consistent with the defini- 
tion of || • If T is a linear functional on (gi^lp-Ej for some Hilbert spaces 
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Ej, and X = {Li, . . . ,L r } G T'i m ij{o}i then let ||T||j denote the norm of T 
as a r- linear functional on ®i = i[®jeLiEj], given by 

(ei, . . . , e r ) h-> T(ei (8) ... (8) e r ). 
Now, denoting tf fc+1 = ©" =1 L 2 (X^), we can apply the above definition 
to H <g> [©f d=1 L 2 (X^ } )] ~ ^lo-^j and use LemmaElto obtain 

e|| £ A ld (x^)\\ <L d P ^(i)-(^m Uh 
*d=l Xe7> /fe+lu{ 0} 



+ L d ^ p i+(i+dcg(D-(fc+2))/2 / Emax || Aid (X (d) MI 2 



id /MX' 
(2) 

This inequality is just the statement of the Lemma, which follows from 
,, associativity" of the tensor product and its ,,distributivity" with respect 
to the simple sum of Hilbert spaces. Indeed, denoting Jk+i = {d}, we have 
for £ L{ and U = Ujgij Jj> 

®^ Li Hj ~ ® jeLt ®ieJi (^ =1 L 2 (XP)) ~ ® /eC/ (©: =1 L 2 (XP)) ~ © M < n X 2 (X 

Similarly, if G Lj, 

© jeLi F, ~ [© |iA .!< n L 2 (Xf-,^)] x [® ^ jeLi ® leJj (®UL 2 (XP))\ 
^© M < n L 2 (X£ c ,tf), 

where C7 = (Uo^jgLi ^i) ^ ^- Using the fact that for fixed xj® , Ai d 
corresponds to the multimatrix (/ii(Xf ec ))| i/d |< n , and A corresponds to 

(/ii(X^ cc ))|i|< n , we can see, that each summand || • ||j on the right hand side 
of ([5]) is equal to some summand || • \\l,K on the right hand side of ([I]). In- 
formally speaking and abusing slightly the notation (in the case K = 0), we 
,, merge" the elements of the partition {{d}, Ji, . . . , J^, K} or {J%, . . . , Jk,K} 
in a way described by the partition X, thus obtaining the partition {L} U/C, 
where L is the set corresponding in the new partition to the set Lj G X, 
containing (in particular, if K = and {0} G X, then L = 0). Let us also 
notice, that deg(X) = deg(/C) + 1, hence 

1 + deg(X) - (k + 2) = deg(/C) - deg^), 

which shows, that also the powers of p on the right hand sides of (H|) and 
([2]) are the same, completing the proof. □ 



dec"! 
it/ > 
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Proof of Theorem^ For d = 1, the theorem is an obvious consequence of 
Lemma[TJ Indeed, since | • | = sup^| <1 \(f>(-)\, and we can restrict the supre- 
mum to a countable set of functionals, we have 

E| HXi)\ P < L p {(n E WiW + f' 2 sup hiXi)) 2 )^ 2 



<1 



+ pPEmax|/i J (X i )| p ). 



But E\^hi(Xi)\ < v^EWF= VE^TOF= ll(^)i||{l},0 
and we also have sup| <)!) |< 1 (^iE((/),/i i (Xj)) 2 ) 1 / 2 = ||(/ij)jlk{i} andmax < \hi{Xi)\ 
maxj 11^110^. 

We will now proceed by induction with respect to d. Assume that the 
theorem is true for all integers smaller than d > 2 and denote I c = I c \{d} 
for I C I d . Then, applying it for fixed xj; to the array of functions 
(E,i d hi{xi,..., Xd-i, , we get by the Fubini theorem 

E| J>(X? CC )P> 

i 

E E ^^^E^kE^jl, 

where we have replaced the maxima in ijc by sums (we can afford this 
apparent loss, since we will be able to fix it with Lemma [2]). Now, from 
Lemma Q] (applied to E^) it follows that 

il\\ P K,J - ^((^IKE h i)h\\K,j) P +P P/2 |l( /l i)i/u{d}H^, 1 7U{{4} 

id 

Since / c = (/ U {d}) c , degJU {{d}} = deg J + 1 and #/ c = #/ c + 1, 
combining the above inequalities gives 

i KCICI d J£V IXK ijc 

+ E E p K#fB - HfaB ^ /a) E%«w(E^ii^) p 

KCICIj.i JeV T \ K ijc id 
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By applying Lemma H] to the second sum on the right hand side, we get 

i KCICI d jePr\ K ijc 

(3) 

We can now finish the proof using Lemma [2j We apply it to E/c for 
J 7^ Id, with instead of d and p/2 instead of p (for p = 2 the theorem is 
trivial, so we can assume that p > 2) and a = 2#I C + deg^7 + #/ c . Using 
the fact that {p/2) a # IC < L p d and E[|(/H)i 7 ||^ >i7 < Ei, E /N 2 5 w e get 



Ijc 



max ||(/ii)i 7 ||^ j- 

ijc 



+ max/«/2 Ej max( J2 Ejc v \\ (h^ \\ 2 K ,j) 

i/ c \.J 

< L p d [Ejc max || (/li)^ H^- ~ 

+ p -(#/ c +de gl 7/2) P maxEj max(^ Ejc\h(X.f ec ^ 2 



\p/2 



= L P d (Ejc max (K^i, ||^ + p -i#i c +W/2)P maxEj max 11(^)^11^ 

which allows us to replace the sums in i/c on the right-hand side of (J3]) by 
the corresponding maxima, proving the inequality in question. □ 

Theorem [1] gives a precise estimate for moments of canonical Hilbert 
space valued [/-statistics. In the sequel however we will need a weaker 
estimate, using the || • \\k,j norms only for I = Id and specialized to the case 
h\ = h. Before we formulate a proper corollary, let us introduce 

Definition 7. Let h: S rf — » H be a canonical kernel. Let moreover X\,X%, . . . 
be i.i.d random variables with values in S. Denote X = (X\, . . . ,Xd) and 
for J C I d , Xj = {Xj) jeJ . For K C I C I d and J = {J 1} ..., J k } E Vi\ K , 
we define 

k 

\\h\\ K ,j = S up{E I (h(X),g(X K ))l[f j (Xj j ):g: Y* K -> H, E\g(X K )\ 2 < 1, 

3=1 

/. . E #Jj ^ M) E/^XjJ) 2 < 1, j = 1, . . . ,k}. 
In other words \\h\\K,j is the \\ • \\k,j of an array (h{)\\\ = i, with hn u = h. 
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Remark For I = I d , \\h\\K,j is a norm, whereas for / C I d , it is a random 
variable, depending on Xjc 

(i) 

It is also easy to see that if all the variables Xf are i.i.d. and for all 
|i| < n we have h\ = h, then for any fixed value of ijc, 

m)\ il \<jK,j=\\h\\ K ,jn* 1 / 2 , 

where ||/i||/£-,.7 is defined with respect to any i.i.d. sequence Xi, . . . ,X d of 
the form Xj = X^ for j £ I c . 

We also have < \/Ki\h(X)\ 2 , which together with the above 

observations allows us to derive the following 

Corollary 1. For all p > 2, we have 

i KCI d j€Pi d \ K 

+ £ p^ +#/C )/ 2 n#^ 2 E /c max(E / |/ i (Xf cc )| 2 )^ 2 ) 
ici d lIC 

The Chebyshev inequality gives the following corollary for bounded ker- 
nels 

Corollary 2. // h is bounded, then for all t > 0, 
P(j^/i(X? cc )| > L d (n d / 2 (E\h\ 2 )V 2 +tj) < 

i 

r 1 / ft \2/dcg(J)x 

/ . / t x2/(d+#/ c ) 



Before we formulate the version of exponential inequalities that will be 
useful for the analysis of the LIL, let us recall the classical definition of 
Hoeffding projections. 

Definition 8. For an integrable kernel h: T, d — ► H, define -ir d h: S fc — ► R 
wii/i i/ie formula 

ir d h(xi, . . . , x k ) = (5 X1 - P) x (S X2 - P) x ... x - P)/i, 

where P is i/ie /aw of X\ . 
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Remark It is easy to see that irkh is canonical. Moreover ir^k = h iff h is 
canonical. 

The following Lemma was proven for H = R in [2] (Lemma 1). The 
proof given there works for an arbitrary Banach space. 

Lemma 6. Consider an arbitrary family of integrable kernels h\ \ T, d — > H, 
|i| < n. For any p > 1 we have 



r dec\ II 

Hp - 

|i|<n N< n 



In the sequel we will use exponential inequalities to [/-statistics gener- 
ated by TTdh, where h will be a non-necessarily canonical kernel of order 
d. Since the kernel h((ei,Xi), . . . , (e d , X d )) = e\ - ■ ■ e d h(X\, . . . , X d ), where 
£j's are i.i.d. Rademacher variables independent of X^s is always canoni- 
cal, Corollary [H Lemma [6] and the Chebyshev inequality give us also the 
following corollary (note that = 

Corollary 3. If h is bounded, then for all p > 0, 
£>^(X? ec )| >L d (n d ^(E\h\ 2 ) 1/2 + t)) < 

i 

1 / ft \2/dcg(J)N 

min — 77—, — r. ) A 



L d \K<zi d ,jar Id \ K \n d / 2 \\h\\ K)J 

t N 2/(d+#I c ) 

A 



L d exp 

. Yet {-^m^hJy^J 

4 The equivalence of several LIL statements 

In this section we will recall general results on the correspondence of various 
statements of the LIL. We will state them without proofs, since all of them 
have been proven in [9] and [2] in the real case and the proofs can be directly 
transferred to the Hilbert space case, with some simple modifications that 
we will indicate. 

Before we proceed, let us introduce the assumptions and notation com- 
mon for the remaining part of the article. 

• We assume that (Xj)j e pj, (-X^)i g fl,i<fc<d are i.i.d. and h: S d — > H is 
a measurable function. 

• Recall that (ei)ieNi ( £ \ )ieN,i<k<d are independent Rademacher vari- 
ables, independent of (Xi) ieN , (X^) ieNt i< k < d . 
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• To avoid technical problems with small values of h let us also define 
LLx = loglog (x V e e ). 

• We will also occasionally write X for (X± , . . . , Xd) and for I Q Id, 
Xf = (Xi)i e f. Sometimes we will write simply h instead of h(X). 

• We will use the letter K to denote constants depending only on the 
function h. 

We will need the following simple fact 

Lemma 7. //E|/i| 2 /(LL|/i|) d = K < oo then E(\h\ 2 A«) < L(loglogii) d with 
L depending only on K and d. 

The next lemma comes from [9]. It is proven there for H = M but the 
argument is valid also for general Banach spaces. 

Lemma 8. Let h: T, d — > H be a symmetric function. There exist constants 
Ld, such that if 

limsup 1 — —pr\ V/i(Xi)| < Ca.s., (4) 
rwoo (raloglogra)*^ f-^ d 

then 

oo 

£>(| ^ ef ec /i(X? cc )| > D2 nd / 2 log d/2 n) < oo (5) 

n=l |i|<2 n 

for D = L d C. 

Lemma 9. For a symmetric function h: T, d — ► -ff, £/ie L7L is equivalent 
to the decoupled LIL 

limsup / |^>(Xf cc )|<Da.s., (6) 

■-- (moglog n) 61 /^ ' 



n— >oo , .- _ • ■ , 



meaning that implies with D = LdC, and conversely |||) implies 
TOi/t C = LdD. 

Proof. This is Lemma 8 in [2]. The proof is the same as there, one needs only 
to replace l°° with l°°(H) - the space of bounded H- valued sequences. □ 

The next lemma also comes from [2] (Lemma 9). Although stated for real 
kernels, its proof relies on an inductive argument with a stronger, Banach- 
valued hypothesis. 
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Lemma 10. There exists a universal constant L < oo, such that for any 
kernel h: T, d — * H we have 



'(max | ^ ^( X f eC )| > *) < L M| S - t l Ijd )- 

i: ik<jk,k=l—d \i\<n 



Corollary 4. Consider a kernel h: T, d ^ H and a > 0. // 

oo 

J>(| £ ^(Xf ec )| > C2™log a n) < oo, 

n=l |i]<2« 

limsup-— r-l V /i(X? 0C )| <L da C a.s. 

- ~ (raloglogn) a 1 ^ v 1 ;| ~ ' 



|i|<2' ; 



Proof. Given Lemma [T0"| the proof is the same as the one for real kernels, 
presented in [2] (Corollary 1 therein). □ 

The next lemma shows that the contribution to a decoupled U-statistic 
from the 'diagonal', i.e. from the sum over multiindices i ^ I d is negligible. 
The proof given in [2] (Lemma 10) is still valid, since the only part which 
cannot be directly transferred to the Banach space setting is the estimate of 
variance of canonical U-statistics, which is the same in the real and general 
HUbert space case. 

Lemma 11. If h: S d — > H is canonical and satisfies 

E(|/i| 2 An) = 0((loglogu) /3 ), 

for some (3, then 

limsup / | Y, MXf cc )|=0a.s. (7) 

n-+oo (moglogn) a /^ ^ 

Corollary 5. The randomized decoupled LIL 

limsup / | £ e? cc /i(X? cc )| < C (8) 

■— (nloglog n) a i z f — • 



n^roo 



\\<n 



is equivalent to ([5]) ; meaning then if ([Sj) holds then so does <[5|) with D = L^C 
and (E|) implies |2|) u>2i/i C = L&D . 
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The proof is the same as for the real-valued case, given in [2] (Corollary 
2), one only needs to replace h 2 by \h\ 2 and use the formula for the second 
moments in Hilbert spaces. 

Corollary 6. For a symmetric, canonical kernel h: H d —> H, the LIL ^ 

is equivalent to the decoupled LIL 'with diagonal' 

lim su p ? i r W2 1 E ^( x i oc ) I < D ( 9 ) 

n^oo (nloglog n) a / z 

\i\<n 

again meaning that there are constants L^ such that if holds for some D 
then so does (0|) for D = LjC, and conversely, (0|) implies for C = L&D. 

Proof. The proof is the same as in the real case (see [2], Corollary 3). Al- 
though the integrability of the kernel guaranteed by the LIL is worse in the 
Hilbert space case, it still allows one to use Lemma [TTJ □ 

5 The canonical decoupled case 

Before we formulate the necessary and sufficient conditions for the bounded 
LIL in Hilbert spaces, we need 

Definition 9. For a canonical kernel h: T, d —> H, K C 1^, J = { J 1; . . . , J^} £ 

Vj d \x and u > we define 



\h\\K,j,u = sup{E{h(X),g(X K ))l[f i (Xj i ): g: T, K — > H, 

fi-. £ Jl — ► R, \\g\\2, ll/ilb < 1, Hfflloo, ll/*lloo <^}, 



where for K = by g(Xx) we mean an element g £ H, and \\g\\2 denotes 
just the norm of g in H ( alternatively we may think of g as of a random 
variable measurable with respect to 0"((-?Q)ie0)j hence constant). Thus the 
condition on g becomes in this case just \g\ < 1. 
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Example For d = 2, the above definition reads as 

\\h(X 1 ,X 2 )\\ mit2}hu = sup{\Eh(X 1 ,X 2 )f(X 1 ,X 2 )\: 

E/(X!,X 2 ) 2 < 1,11/Hoo <«}, 
\\h(X 1 ,X 2 )\\ U{1}{2}hu = S up{\Eh(X 1 ,X 2 )f(X 1 )g(X 2 )\: 

Ef(X 1 ) 2 ,Eg(X 2 ) 2 < 1 

ll/lloo, Hfflloo < U}, 

\\h(X 1 ,X 2 )\\ {lh{{2}hu = S np{E(f(X 1 ),h(X 1 ,X 2 ))g(X 2 ): 

E|/(Xi)| 2 ,E^ 2 ) 2 < 1 

ll/lloo, Halloo, < u} 

\\h(X 1 ,X 2 )\\ {h2}Au = S up{E(f(X 1 ,X 2 ),h(X 1 ,X 2 )): 

E|/(Xi,X 2 )| 2 <l, ll/lloo <u}. 

Theorem 2. Let h be a canonical H -valued symmetric kernel in d variables. 
Then the decoupled LIL 



^P ^nJ^Ws l E ^Xf cc )l < C (10) 

i|<n 



n d / 2 (loglog n) d / 2 
ZioWs i/ and only if 



and for all K C J dj ,/ G Vj d \K 

hmsup— J d _ dcg 7 w 2 IMkj> < D. (12) 

«->oo (loglogn)V d 

More precisely, if A10\) holds for some C then is satisfied for D = L^C 
and conversely, [T7\) and (TOfy implies I70j) with C = L^D. 

Remark Using Lemma [7] one can easily check that the condition (|12p with 
D < oo for I = Id is implied by pip . 



6 Necessity 

The proof is a refinement of ideas from [16], used to study random matrix 
approximations of the operator norm of kernel integral operators. 
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Lemma 12. If a,t > and h is a nonnegative d- dimensional kernel such 
that N d Eh(X) > ta and ||E//i(.X')|| 00 < N'^a for all C I C {l,...,d}, 
then 

V A e (o,i) n E ^( X i° C ) ^ Ata ) ^ ( 1 ~ A ) 2 t + 2 d_ 1 ^ (l-A) 2 2- d min(l,t). 

|i|<7V 

Proof. We have 

e( £ Mxf cc )) 2 = E E EMxf cc )Mxf c ) 

|i|<7V ' |i| <7V |j | <JV 

= E E E EMxf ec )Mxf c ) 

IQId\i\<N \i\<N: 

{k- ik=jk}=I 

= E E E E[Mxf cc )E /c Mxf c )] 

|i|<AT |j|<JV: 

{fc: ik=3k}=I 

< N 2d (Eh(X)) 2 + ^ d+# - rC E^(X)||E / c/i(X)|| 0O 

< iV M (E/i(X)) 2 + (2 d - l)N d aEh(X) 

< N 2d (Eh(X)) 2 + (2 d - l^N^iEhiX)) 2 

< t -±^i(E^h(xt- ) y. 

[i[<n 

The lemma follows now from the Paley-Zygmund inequality (see e.g. [5], 
Corollary 3.3.2.), which says that for an arbitrary nonnegative random vari- 
able 5, 

F(S>XS)>(l-X) 2 ^. 

□ 

Corollary 7. Let A C £ rf be a measurable set, such that 

V c/c { i,...,4 v ^ e ^ ^~ #/ - 

T/ien 

P(3| iKJV Xf ec G A) > 2- d mm(N d F{X G A), 1). 
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Proof. We apply Lemma QJ with h = I a, a = 1, t = N d P(X E A) and 
A^0+. □ 

Lemma 13. Suppose that Zj are nonnegative r.v. 's, p > and a,j E K are 
smc/i i/iai P(2j- > (ij) > p for all j. Then 

j j 
Proo/. Let a := PQ^ Zj > pEj a j/ 2 )> then 

j j j j 

□ 

(i) 

Theorem 3. Let Y be a r.v. independent of X\ . Suppose that for each n, 
a„£l, h n is a d + 1- dimensional nonnegative kernel such that 

J>( Y, h n (X.f cc ,Y)> an ) <oo. 

n |i|<2 n 

Let p > 0, then there exists a constant Cd(p) depending only on p and d 
such that the sets 

A n := {xeS d : V n < m < 2d - ln F Y (h m (x,Y) > C d (p)2< n - m ^a m ) > p}, 
satisfy £ 2 dn F(X E A n ) < oo. 

Proof. We will show by induction on d, that the assertion holds with C\ (p) := 
1, C 2 (p) := 12/p and 



,,-J fo-l-i 

For d = 1 we have 

I - - - • . , . • - — ~T<UV 

■-dec 



C d (p) := Up" 1 max Q.^"'-^^) for d = 3, 4, . 

KKd- 1 



2 min(2"P(X E A n ), 1) < P(3| i |< 2 , l Xf ec e A r 



= P(3|i|< 2 n P y (/i n (Xf ec ,y) > a n ) > p) 
<F(F Y (Y h n (Xf cc ,Y)>a n )>p) 

|i|<2 n 

<p- X P(E /in(Xf ec ,y)>a n ). 



|i|<2 n 
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Before investigating the case d > 1 let us define 

A n '.= A n \ A m . 

The sets A n are pairwise disjoint and obviously A n C A n . Notice that 
since Cd(p) > 1, 

r(X E A n ) < F(F Y (h n (X,Y) > an)) < Y, ^n(Xf cc ,y) > On). 

|i|<2" 

Hence E n F ( X e A «) < °°> so P ( X G limsupA n ) = 0. But if x (£ 
limsupA n , then J2 n 2 nd I A n (x) < ^2 n 2 nd+1 1 ^ ( x )- So it is enough to show 
that £2 dn P(X G A n ) < oo. 

Induction step Suppose that the statement holds for all d! < d, we will 
show it for d. First we will inductively construct sets 

A n = A n D A n D . . . D >l n 
such that for 1 < Z < d — 1, 

V c/c { i,...,d-i}, #/</ V* JC P/((x/e, X/) E < 2~™# 7 (13) 

and 

^2^P(XG<- 1 \<)<oo. (14) 

n 

Suppose that 1 < I < d — 1 and the set ^4^ _1 was already defined. Let 
J C {1, . . . , d} be such that = I and let j G I. Notice that 

Pj((sjc,Xi) G <- X ) = Ej-P^-jfCxie,^,^}) G A^ 1 ) < 2-™( / ~ 1 ) 
by the property (fT3"|) of the set A^ -1 . Let us define for n(l — 1) + 1 < k < nl, 

B* ntk := {xjc-.Fjdxj^Xj) G A 1 ' 1 ) G (2" fc ,2- fc + 1 ]} 

and 

ni 

== U B n,fc = = Pj(0^ */) G A^ 1 ) > 2~ nl }. 

fc=ra(Z-l)+l 
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We have 



nl 

2 dn ¥(X G A l -\X IC e^)<2^ 2 dn ^ fc F(X /c G B T n ) 

n n k=n(l—l)+l 

= 2Efc{(X/c), 

where 

nl 

^(^)-=E E 2"/ B , (x /c ). 

n fc= n (i-l)+l 

Let m > 1 and 

:= {x /c : 2( m+1 ^) > fc^x/c) > 2 m ^}. 
Notice that for n > m and A; < nl, 2 dn ~ k > 2( d ~0( m + 1 ) j moreover 



m 49 
- - 3 - 3 

n<rn/2 k=n(l-l)+l n<m/2 

Hence 



x /c GCi^ £ £ 2 rf «- fe / <fc (x /c ) > (15) 

m/2<n<m fc=n(Z-l)+l 



Let m < r < 2 d 2 m, if m/2 < n < m, then since ^ 1 C A n we have for all 
x G 

Py(/i r (x, Y) > C d (p)2 d ("™ r )a r / A; „ 1 (x)) > p, 
therefore, since C A n are pairwise disjoint, 

Py(/i r (x,y) >C d (p)2- dr a r 2 d "/^_ 1 (x)) >p. 

m/2<n<m 

Hence, by Lemma [T3l 



V( E h r (xj.,X^,Y)>^C d (p)2- dr a r £ ^ c (Xf; c )j > |, (16) 

\il\<2 r M<2 r 
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where 



k 2 , Xl c{xi) := E 2 dn I A i-i(x I c,x I ). 

m/2<n<m 

We have ||A^, Xjc ||oo < 2 dm and for ^ J C I, by the property (HSJ) of 



Ejfe2, Xjc (x A j,Xj)= ^ 2*P J ((o; I c,x AJ) l J ) G^" 1 

m/2<n<m 

< ^ 2 { - d ~* J)n < 2 (d -* J ^ m+1 . 

m/2<n<m 

Moreover for x/c € , by the definition of k and ([To]) , 



nl 

m, Xlc ( X l)> E E 2 dn ¥ I ((x I c,X I )eA l - 1 )I Bl (xjc) 

m/2<n<m k=n{l—X)+l 

nl 1 

> ^ 2 dn - k I B i k (x IC ) > - 2 ^ m . 

m/2<n<m fc=n(Z-l)+l 

Therefore by Lemma PT21 (with / instead of d and a = 2( d ~^ m+rl+1 , t 
1/6, JV = 2 r , A = 1/2), for m < r < 2 d ~ 2 m, 



£ ^ IC (xf; c )>i 2 ( d - 



|i/|<2 r 



~ 3 



Combining the above estimate with (|16p we get (for x/c g and m < r < 



2 a ~ 2 m), 



E M^,xf; c ,y) > ^c d ( P )2( d -w m -'V) > ^-'~V 

|i/|<2 r 

Let us defined := {(X^) je i,Y) and h n (xic,Y) := E| i/ |< 2 « (^ , X?; c , F ) . 
Then 

E>( ^ Mx?; c c ,y) > o„) = E p ( E M x ? cc ^) > <0 < °o. 

n |i/ c l<2 n n |i|<2 n 
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Moreover (since C d {p) > 12p- 1 C7 (i _/(2~ I ~ 4 p/3)), 

Cl C {v m < r < 2d -2 m FyCh r ( Xl c,Y) > C d _ 1 (2- i -V3)2 ( " )(m - r) a r ) > 2~ l - A p/'i}. 
Hence by the induction assumption, 

J22^ m ¥(X I c G C^) < oo, 

m 

so E/s{(X/c) < oo and thus 

V#i=i £ 2 " np (^ e 4r\*/' G «n) < oo- (17) 

n 

We set 

A l n := {x G : x /e £^ for all / C {1, . . . , d}, #/ = I}. 

The set A l n satisfies the condition by the definition of B n and the 
property dTSJ for A 1 ' 1 . The condition dHJ) follows by ([XT]). 

Notice that the set ^4^ _1 satisfies the assumptions of Corollary [7] with 
iV = 2 n , therefore if C d {p) > 1, 

2- d min(l,2" d P(X G ^f 1 )) < P(%|<2» Xf cc G A^ 1 ) < P(3|i|< 2 „ Xf cc G i n ) 

< P(3 |i| < 2 „ Py(/ ln (Xf ec ,y) > C d (p)a n ) > p) 
<P(Py(^ / i „(Xf cc ,y)>a n )>p) 

|i|<2 n 

<fME / ln (xf cc ,y)>a n ). 

|i|<2 n 

Therefore J^ n 2 nd ^{X G < oo, so by (HU) we get 

d-l 

2 nd P(x G An) = J2 ^(Y, ¥ ( X G A n X \ A) + £ ^n" 1 )) < °°- 

n n Z=l 

□ 
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Corollary 8. If 

Y, F (Y, ^ 2 ( X f eC ) > e2 nd (logn) Q ) < oo 



n i|<2™ 

for some e > and q£R, 

Proof. We apply Theorem [3] with /i n = /i 2 and a n = e2 nd log rf n in the 
degenerate case when Y is deterministic. It is easy to notice that h 2 > 
C d (p, e)2 dn \og d n implies that 

V n < m < 2d -i n h 2 > C d (p)2 d ^a m . 

□ 

To prove the necessity part of Theorem [2] we will also need the following 
Lemmas 

Lemma 14 ([2], Lemma 12). Let g: T, d — > R 6e a square integrable function. 
Then 

Var(^ 5 (Xf cc )) < (2 d - l)n M - x E 5 (X) 2 . 

i <n 

Lemma 15 ([2], Lemma 5). // E(|/i| 2 A u) = C((loglog uf) then 

nm m> _, } = 0{ <Wf^). 

Lemma 16. Let (ai) ieI d be a d-indexed array of vectors from a Hilbert space 
H . Consider a random variable 



,! 



sHE- IK' = IE< 



,dec 



|i|<n k=l |i|<'« 

For any set K C I d and a partition J = {J%, . . . , J m } £ Vj d \K let us define 

m 

\i\<n k=l i K ij k 



Vi , e / n V(4 fe) ) 2 < 1, * = l,...,ml, 
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where oj = J\{max J} (here a\ = a\). 
Then, for all p > 1, 



s\\ P >±- £ H(«i)HW 

d Kci d ,jeV Id \ K 



In particular for some constant Cd 

HS>c d £ \\(ai)\\*K,j, P )>c d Ae-P. 

KCI d ,JeP Id \ K 

Remark For K = 0, we define 

m 

|i|<n fc=l i Jfe 



It is also easy to see that for a d-indexed matrix, ||(ai)i||j d) {0} iP = yf^li l a il 2 
||5||2 and thus does not depend on p. Since it will not be important in the 
applications, we keep a uniform notation with the subscript p. 

Examples For d = 1, we have 

n n 

\\( a i)i<n\\l t{ {i}}. p = sup{| ^2aiO>i\ : ^a- <p, \a>i\ < 1, i = 1,. . . ,n}, 

i=i i=i 



(ai)i<„ ||{ 1}j0iP = sup I ^2( ai ,ai): £ H 2 < i} = , £ k 

\ i=i 



n 

12 
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whereas for d = 2, we get 

n n n 

\\Mij<n\\t,{{l},{2}}# = SU P { I Y a iJ a ^j I : Y °t ~ P > Y $ ~ P i 

i,j=l i=l j=l 

VieJ n |oi| < l,Vj- e / B |/3,-| < l}, 

n n n 

l|(«y)ij<n||0,{/ 2 }, p = sup{| a ij(*ij\- Y a % - P^3eI„Y a iJ - ^ ■ 



i=l 



( a y)ij<n|l{l},{{2}},p = SU P{| Y\ ai \ 2 - 1 ' 

i,j=l i=l 

X;^<p,v^ Jb i^i <i}, 

n n 



(aij)i,i<n||/ 2i0i p = SUp { I Y ( a iji a ij)\ : a \ < !} = . E I 

M=l ij'=l V ij 



,..|2 



Proof of Lemma [Tb\ We will combine the classical hypercontractivity prop- 
erty of Rademacher chaoses (see e.g. [5], p. 110-116) with Lemma 3 in [2], 
which says that for H = R we have 

\\s\\ P >±- Y \\( a 0h,j, P - (is) 



Since ||(ai)||/ d ,{0}, p = VEi l«i| 2 = 1 1 # lb, the inequality ||5|| p > L 1 \\(a i )\\ Id ^ }!P 
is just Jensen's inequality (p > 2) or the aforesaid hypercontractivity of 
Rademacher chaos (p E (1,2)). On the other hand, for K ^ 1^ and 
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J G 'Pl d \K, we have 



P\ 1/p 



ijC fc£-R" fcf^ 

|2\ p/2 N 1/p 



> 



L 



#K 



E 



Id\K 



e|e«.il 



l-(e w , sup |EE(*»i>n 



> 



1 



-(*) 



p\ 1/p 



^'K 1 'A' 1 - 



> 



sup E Id \ K | £ Yl Kc ' II e * 

(0) 



P\ 1/P 



L^K^d-#K 



sup -«i))i/ d \A- 



0,J,p 



p' 



where the first inequality follows from hypercontractivity applied condition- 
ally on (£i^)k&K,iei n i ^ ne secon d is Jensen's inequality and the third is (|18|) 
applied for a chaos of order d — #K. 

The tail estimate follows from moment estimates by the Paley-Zygmund 
inequality and the inequality ||(ai)||.fr„7,tp < t \\(fli)\\K,j,p for t > 1 just 
like in [El H]. □ 



Proof of necessity. First we will prove the integrability condition (lllh . Let 
us notice that by classical hypercontractive estimates for Rademacher chaoses 
and the Paley-Zygmund inequality (or by Lemma [16]) . we have 



£ e ? ec Mx? ec ) > Cd / J2 Mxf ec ) 2 ) > c d 



|i|<2 



for some constant c d > 0. By the Fubini theorem it gives 

P £ (| ef ec /i(Xf cc )| > D2 nd ' 2 log d / 2 nj > c d p( ^ /i(Xf 0C ) 2 > D 2 c d 2 2 nd log° 

|i|<2™ ' |i|<2 n 

which together with Lemma [8] yields 

J^P( ^ h(X.f cc ) 2 > D 2 cf2 nd log d nj < oo. 

n \i\<2" 



n 
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The integrability condition (jlip follows now from Corollary [8l 

Before we proceed to the proof of (fT2|) , let us notice that (fTT|) and Lemma 
[7] imply that 

E(|/i| 2 Am) < ^(loglog u) d (19) 



for n large enough. The proof of (|12j) can be now obtained by adapting the 
argument for the real valued case. 

Since lmin^oo Y^kLn \ = 1°§ 2, (JSJ) implies that there exists iVo, such that 
for all N > No, there exists N < n < 2N, satisfying 

ef ec /i(Xf ec )| > L d C2 nd ' 2 \og d/2 n) < -p (20) 



|i]<2™ 

Let us thus fix TV > iVo and consider n as above. Let K C. 1^, J~ = 
{ Ji, . . . , J k } G Let us also fix functions 5: S*^ -» if, /j : S #J J -> R, 

j = 1, . . . , k, such that 

\\g(x k )\\ 2 < 1, \\g(x K )\\oo <2"/( 2fc+3 ), 
||/,(x Jj )|| 2 <i,||/ J (x Jj )|| 00 <2"/( 2fc+3 ). 

The Chebyshev inequality gives 

P( / J (X^ c ) 2 logn<10-2 d 2#^logn)>l- I ^ J . (21) 

|i^l<2" 

Similarly, if K ^ 0, 



£ bCxg^)! 3 < 10 . j^^-) > 1 - 



(22) 



M<2" 



and for if = 0, |g| < 1 (recall that for K = 0, the function 5 is constant). 
Moreover for j = 1, . . . , k and sufficiently large N, 

V- 1 -, v d«N2 , 2™#^2 2 «/( 2fc+3 )logn 
> — jrrfiPH* ) • logn < — 

|WJ<2" 



3 ' 



2 2n/(2fc+3) !„„ 
< 5— < 1. 

Without loss of generality we may assume that the sequences (X^ 
and (e[ j) )i a are defined as coordinates of a product probability space. If for 
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each j = 1, . . . , k we denote the set from (j2T|) by A^, and the set from (f22|) 
by A , we have P(D^=o^fe) ^ °- 9 - Reca11 

now Lemma [TBI On P|j=o^ fc we 
can estimate the || • \\* K ji ogn norms of the matrix (/i(Xf cc ))|i|< 2 n by using 
the test sequences 

/,(xf; c )Vioi^ 



for j = 1, . . . , k and 



a 



KTj 10 l/2 2 d/2 2 n#J j /2 



(0)_ 5(X£) 



IK iQl/22d/22n#K/2 ' 

Therefore with probability at least 0.9 we have 



ll(M X f eC ))|i|<2™ll!f,J,logn ( 23 ) 

- 2 d(fc+i)/2 10 (fc+i)/2 2 (#A:+E J - #J>/2 1 Y.y(^ c ),h(X.f cc )} n/,(Xf« c )| 

^^^/^ ^ f~I 

= 2 d(fe+l)/2in(fc+l)/22rfn/2 ' ^(Xf°°), /l(Xf 6C )) JJ/j(Xf^ C )|. 

|i|<2™ i=l 

Our aim is now to further bound from below the right hand side of 
the above inequality, to have, via Lemma [TBI control from below on the 
conditional tail probability of Yl\i\<2 n tf ec h(X.f ec ) , given the sample (X^). 

From now on let us assume that 

k 

\M(g(X K ),h(X))llMXj s )\>l. (24) 

3=1 

The Markov inequality, (|19|) and Lemma [T5l give 

2^\E(g,h) nJU/i 1 



p(i x; (5(x i ,),/ l (xf-))i m(xfcc)l>2n} n/ J (x^ c )i > x 4 /iij=i ^ 1 ) 

|i|<2" J=l 

< ^(ll^lloo n?=l Halloo) • E|fa|l { | A>2 „ > < 42 n( fc+ l)/(2, + 3) E|/l|1 

<4^^. (25) 

2 2fc+3 
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Let now h n = W-{iw<2' 1 }- By the Chebyshev inequality, Lemma [13] and ([19]) 



p i e M**? )Mx? ec )) n .^ x! 'r j - 2ndE ^' ^ n /: 

V |i|<2 n 3=1 j=l 



k 

4 J\ 



2nd k 
> 

" 5 



n9,K)Uf3\ 



" 25 ^\E(g,h n )UUM 2 

fnd _ -i\r)n(2d— 1) j* 

< 25 ^ -^— h E\(g,h n )T\ fA 2 

^ 25(2 d 2 2 ^ + l)/(2 fc+ 3) E | fen |2 
2 n |E(5,/in)n)=l/j| 2 

< 25^(2^ - 1)— lQg n , . (26) 

Let us also notice that for large re, by ([19]) . Lemma [TBI and ([23]) 

k k k 

ie<5, m n /ii ^ i e ^' ^ n /ii - i e <s, /»>i{i/.i>2»} n &i 

3=1 3=1 3=1 

> |Efc,A> fl/il - 2 ^)/(^)A'i^ > ||E^,ft> n/ii > |. 

3=1 3=1 

(27) 

Inequalities ([25]) . ([26]) and ([27]) imply, that for large re with probability 
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at least 0.9 we have 

k 

l ^( 5 (xf-),/ l (xf ec ))n/i(xf; c )i 

|i|<2™ j=l 

k 

> | £ ( 5 (xf-),/ ln (xf ec ))n/i(x5 c ) 

|i|<2' 1 j=l 



-i ^ ( ff (xf-),/ l (xf cc ))i m(xfcc)l>2 „ } n/,(x^)i 

|i|<2 n 3=1 

A n 1 ^ 

> 2" d (-|E< 5 , ^) J] /il - t|E<<7, h) J] /il) 

A K n 1 ^ onei ^ 

> 2 ^(- . |ie^ fc> n - >( 5j h) n /,-D > — ft> n /. 



3=1 3=1 3=1 



Together with (|23p this yields that for large n with probability at least 

0.8, 

2 nd/2 bg fc/2 n * 

ll(^i)[i[<2»||^,logn > 4 . 2 <i(Jfc+l)/2 10 (fc+l)/2l E ^' /t / ll^'l- 

3=1 

Thus, by Lemma [16] f° r large n 

ond/2 r fc/2 fc o 

p(i x; ^(xf»)i > c, ^ iy ^ iy2 m,h)Y[ fj \) > — , 

|i|<2« " 3=1 



which together with (|20p gives 



iE<», fc> n m ^ LdC 4 - 2d(fc+i)/2i ° (fc+i)/2 log(d -*)/ 2 n . 

In particular for sufficiently large N, for arbitrary functions g: Yft K — > ff, 
/j : R, j = 1, . . . , k, such that 

||s(x*)|U 11/^x^2 <i, 

|| 5 (Xx)|| 2 ,||/ J (X Jj )|| 00 <2 Ar /( 2fc + 3 ) 

we have 

\E(g, h) f[ /il < LdC 4 -^ 1)/2loM/2 logC^/a n < Z d Clog^ fc )/ 2 iV, 
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which clearly implies (|12p . □ 

7 Sufficiency 

Lemma 17. Let H = H(X\, . . . , Xj) be a nonnegative random variable, 
such that EH 2 < oo. Then for I C I d , I ^ ; I d , 

oo oo 

^^2 i+#/Cn P /c (E 7 F 2 > 2 2l+#ICn ) < oo. 

Z=0 n=l 

Proof. 

In In 

< ^2 1 ~ l E I cE I H 2 < 4EH 2 < oo. 
i 

□ 

Lemma 18. Let X = and X(J) = ((Xi) i6J , (X t W ) ie jc). De- 

note H = |/i|/(LL|/i|) d / 2 . IfE\H\ 2 < oo and h n = hl An , where 

A n C {x: \h{x)\ 2 < 2 nd log d n andV Wd EjH 2 < 2*^}, 

then for I Q Id, I we have 

n~n#I 

-^-EWKiX^K (X(I))\ 2 } < oo. 

77. O 



Proo/. a) I = Id 



y E|/l "! < Ei/ii 4 V - i 

Z^ 2 ndl 0g 2d n - I I Z^ 2 „d log 2d n 



{|/i| 2 <2" d log d n} 



b) / ^ 2^, 0. Let us denote by E/. E/<-, E/.- respectively, the expectation 
with respect to (Xj)j g /, (Xj)j 6 / C and (X^)i e jc Let also /i, h n stand 
for h(X.(I)), /i n (X(/)) respectively. Then 
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2 n # I log 2d n ~ ^ 2 n # I \og 2d n 



{\h\<\h\}> 



< 2K^\h\ 2 \h\ 2 l 



{\h\<\h\} 

Yl 2 n#I] n „2d - 1 {E / c|/ l |2l {|h|2 < 2 2™d } <L d 2#l™ \og d n, \h\><2^d^) 



2 n#I log 2d n ""{E.c |ft| 2 l {|h|2 < 2 2nd } <i d 2#^ log d n, H 2 ^ 2 -*} 
{N<H} 

X] 2 n#/ log 2d ~ 1 {E /C |fe| 2 l { | h |2<|h|2 } <L d 2#^ log d n, |fc| 2 <2 2 ™ d }) 

1 



< 2E(j/i| 2 |/i| 2 l 



l " |2E7C ^ |2l ^ l ^ l} (MH^l {l ,i |2} )(LL|^|)^ 



|/i| 2 

< L^E ^— L — < oo, 

" (LL\h\r 

where to obtain the second inequality, we used the fact that 

E / c l /l | 2l {|/i| 2 <2 2 " d ,E /c H 2 <2# 1 ™} 

< L d E IC H 2 l {E[cH 2< 2# i n} \og d n < L d 2# In log d n. 



□ 



Lemma 19. Consider a square integrable, nonnegative random variable Y. 
Let Y n = Yl Bn , with B n = \J keK ( n )C k , where C , C\, C 2 , ■ ■ ■ are pairwise 
disjoint subsets of £2 and 

K(n)={k<n:E(Y 2 l Ck )<2 k ' n }. 

Then 
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Proof. Let us first notice that by the Schwarz inequality, we have 
Y E(Y 2 l Ck )) 2 = (( Y 2^/H k -^' 2 ^{Y 2 l Ck ) 

keK(n) keK(n) 

< Y [2"^(E(y 2 l Cfc )) 2 ]^2 fc - n 

k€K(n) k<n 

= 2 Y [2«- fc (E(Y 2 l Cfc )) 2 ]. 

keK(n) 



Thus 



Y^nf^Y 2 [2 n - k (nY 2 lc k )f] 

n n k€K(n) 

<2 y (nY 2 i Ck )f Y T ~ 

k: E(y 2 l Cfc )>0 n: fceA'(n) 

<4 Y ( E ( y21 ^)) 2 ma * 2 

z -— ' n: keK(n) 

k: E(F 2 l Cfe )>0 

fc:E(y 2 l Cfc )>0 V fcy 

< 4^E(y 2 l c J = 4EF 2 < oo. 



n—k 



□ 



Proof of sufficiency. The proof consists of several truncation arguments. 
The first part of it follows the proofs presented in [11] and [2] for the real- 
valued case. Then some modifications are required, reflecting the diminished 
integrability condition in the Hilbert space case. At each step we will show 
that 

oo 

Y F (\ Yl ^n(X? ec )| > C2 nd / 2 log^ 2 n) < oo, (28) 

n=l |i|<2 n 

with h n = hlA n for some sequence of sets A n . In the whole proof we keep 
the notation H = \h\/(LL\h\) d / 2 . 

Let us also fix S (0, 1), such that the following implication holds 

V n= i, 2 ,... \h\ 2 < rf d 2 nd \og d n =^ H 2 < 2 nd . (29) 
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Step 1 Inequality ([28]) holds for any C > if 

A n c{x:\h(x)\ 2 > V 2 d 2 nd log d n}. 

We have, by the Chebyshev inequality and the inequality E|7Trf/i n | < 2 d E|/i„| 
(which follows directly from the definition of ir^ or may be considered a 
trivial case of Lemma [6]) , 

J>(| Yl ^n(Xf cc )| > C2^ 2 log rf / 2 n) 
n |i]<2™ 

E|£]i]<2«^n(X? ec )| 



oo 



C2 n <V 2 log d/2 n 

71=1 ° 



< ^ 2 wd E|/t|l {|fe|>??d2nd/2logd/ 2 n} 
- C2^/ 2 log d/2 n 

= 2 d C- 1 E(\h\ ^^{l^^r^log^n} 
Ifil 2 



Step 2 Inequality ([28]) holds for any C > if 

A„ C {x: |/i(x)| 2 < V 2 d 2 nd log d n, 3 Wd E 7 # 2 > 2# /c "}. 
As in the previous step, it is enough to prove that 



E 



E|£|i|< 2 ^? ec uxf cc )| 



2 nd/2 l og d/2 
n=l ° 



< OO. 



The set A n can be written as 

A n = |J A n (I), 
ICI d ,I^I d ,$ 

where the sets A n (I) are pairwise disjoint and 

^{1) C {x: |/i(x)| 2 < 2 2nd , E/ff 2 > 2 #/Cn }. 

Therefore it suffices to prove that 

A E| E|i|< 2 n e^h(XtniA n (i)(Xf cc )\ 

> — ; J7^ < OO. (30) 

2 nd / 2 \og d ' 2 n 
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Let for I £ N, 

A n j(I) := {x:\h(x)\ 2 <2 2nd , 

2 2 ' +2+#/c ™ > EiH 2 > 2 2l+ # ICn } n A n {I). 

Then h n l An ^ = Y^loK,h where h U; i := h n l An l ^ (notice that the sum is 
actually finite in each point x E S rf as for large I, x ^ A n j(I)). 
We have 

n E ^m^i < E E ' cE 'i E 4 ec/i -,Kxf oc )i 

|i|<2™ |i/o|<2 n |ij|<2 n 

< e e he/I E 4r h n,i(*? ec )\ 2 ) 1/2 

|i/c|<2" |i/|<2 n 

< 2(# /c+ # / / 2 )™E /c (E / |/ in , i | 2 ) 1/2 

< LdpW+^+^log^njFjciEjH 2 > 2 2l+ # ICn ), 

where in the last inequality we used the estimate 

IE/Zl 2 ^ <-L d E/[(l0g n) d i? 2 l{ 2 2!+2+#I c n >E/ #2> 2 2!+#/ c n}] 

<L d 2 2l+2+ # ICn (\ogn) d l {ElH ^ l+ ^ n} . 
Therefore to get (|30p it is enough to show that 



E E 2' +#7Cn P/c(E/ff 2 > 2 2i+ # /c ") < oo. 



1=0 n 

But this is just the statement of Lemma [T71 



Step 3 Inequality ([28]) holds for any C > if 

A n C {x: \h(x)\ 2 < V 2 d 2 nd log d n, V Wd EjH 2 < 2* ICn } n |J 



with Bi = U keK{I , n) Cl and Ctf = {x: E^ 2 < 1}, C{ = {x: 2* IC (k-l) < 

E/# 2 < 2 #/cfc }, ft > 1, iT(J,n) = {k<n: E{H 2 l c i) < 2 k ~ n }. 

By Lemma [6] and the Chebyshev inequality, it is enough to show that 



E — - ■- »■> < x - 



2 2nd l Qg 2d n 
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The Khintchine inequality for Rademacher chaoses gives 
L^E\ ef cc h n (Xf cc )\ 4 <E(J2 IMX? CC )| 2 ) 2 

|i|<2 n |i|<2 n 

= EE E E|^ n (xf ec )i> n (xf-)i 2 ] 

7C7 d |i|<2™ |j|<2«: 

{k: ik=jk}=I 

< 2 nd 2^-# / )E[|/ ln (X)| 2 • |h n (X(/))| 2 ], 

where X = . ..,X d ) and X(J) = (pQ)ie/, (*i (1) )ieJ<0- 

To prove the statement of this step it thus suffices to show that for all 
IQId, 

^ 2~ n # 7 

S W ■= E -^rn\h n (X)\ 2 \h n (X(I))\ 2 } < oo. (31) 

The case of nonempty I follows from Lemma [THJ It thus remains to consider 
the case 7 = 0. Set H] = E/i? 2 . We have 

m = E { ^rr = E^r^ 1 ^ 2 < ^E( E (# 2 uJ) 2 

n lo S " n log " n 

< L d ^(E(F 2 £ 1 B ,)) 2 < Z d ^ £(E(tf 2 l fl ,)) 2 

= L d E Ew*/ 1 **)) 2 < °° 

/g/ d n 

by Lemma [T9l applied for Y 2 = E/if 2 , since E# 2 = EH 2 < oo. 

Step 4 Inequality ([28]) holds for some C < L d D if 

A n = {*: ^(x)! 2 < 7,22 nd log" n, V Wd E^ 2 < 2#' c "} n f| (B n 



where B n is defined as in the previous step. 

Let us first estimate 1 1 (-Kj- 1 ^-n j 2 ) 1/>2 1 1 oo for I C I d . We have 



E/|/i n | 2 < E 7 



\ h \ 2l {\h\ 2 <ri d 2 nd log d n} E 1 

k<n,k(£K(I,n) 

<L d \og d n Yl ± C 'W 2 - 

k<n,k<£K(I,n) 
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The fact that we can restrict the summation to k < n follows directly from 
the definition of A n for I ^ and for / = from ([29)) . 
The sets C[ are pairwise disjoint and thus 

||E/|/g 2 ||oo < (L d \og d n) max 2* rk = L d 2* ICk ^ log d n, (32) 

k<n,MK(I,n) 



where 



k/(n) = max{/c <n:k^ K(I,n)}. 
Therefore for C > 0, 

^ r 1 / C2 nd / 2 \og d/2 n \2/(<i+#/ c ) 
^ 6XP L ~T d \2#^\\(E I \h n \ 2 )^\\ O0 ) 



^ S Z ex p 

n k<n,k(£K(I,n) 

= J2 Yl ex p 

k n>k, k£K{I,n) 



C2 nd / 2 \og d/2 



n 



2/(d+#r) 



L d \2#In/2 2 #I-k/2 l Qg d/2 

j_(c 2 #nn-k)/2y^#^ 



Notice that for each k the inner series is bounded by a geometric series 
with the ratio smaller than some q d> c < 1 (Qd,c depending only on d and 
C). Therefore the right hand side of the above inequality is bounded by 



K y. sup exp 

k n>k, k(£K(I,n) 



J_( C2# nn-ky2Y^#^ 



with the convention sup0 = 0. But k K(I,n) implies that 2# /c ( n k ^ 2 > 
(E(H 2 l c i))~# IC I 2 . Therefore the above quantity is further bounded by 

k 



K E exp[-i-(c-/-E(^l c ,))- /(W) 



<L d C~ 2 l# IC Y^i H2l cl, 



L d C~ 2 /* IC EH 2 < 



OO, 



where we used the inequality e x > c d x a for all x > and < a < 2d. We 
have thus proven that for all I C. I d and C, L d > 0, 



J2 exp 

n: A n ^% 



1 / C2 nd / 2 log d/2 n \2/(d+#n 



L d \2#^/ 2 \\(Ej\h n \ 2 y/ 2 \ 



< oo. 



(33) 
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Now we will turn to the estimation of \\h n \\ j 0t j- Let us consider Jo C 1^, 
J = { Ji, . . . , J{\ £ 'PiAJq an d denote as before X = (X\, . . . , JQ), Xj = 
(Xi)i e i. Recall that 

l 

\\K\\j ,j = sup{E(h n (X),f (Xj ))Hf i (Xj i ): E|/o(X Jo )| 2 < 1, 

i=l 

Ef^X, h )<l, i>l}. 

In what follows, to simplify the already quite complicated notation, let 
us suppress the arguments of all the functions and write just h instead of 
h(X) and /, instead of /^(XjJ. 

Let us also remark that although /o plays special role in the definition 
°f II ' IUoiJ") m what follows the same arguments will apply to all /Vs with 
the obvious use of Schwarz inequality for the scalar product in H. We will 
therefore not distinguish the case i = and ff will denote either the usual 
power or (/o,/o), whereas ||/i||2 for i = will be the norm in L 2 (H, Xj ), 
which may happen to be equal just H if Jo = 0. 

Since E|/j| 2 < 1, i = 0, . . . , I, then for each j = 0, . . . , I and J C Jj by 
the Schwarz inequality applied conditionally to Xj \j 

i 

mKjo)Y[f^{Kjff>^}\ 

i=i 

i 

^ E M4(%\^n/i) i/2i {wi>« 2 }(%v)*i 2 ) i/2 ] 

i=0 

<T T 9 k (./,\J) c ( n )#( J A J )/ 2 r/TR- f2\ l/2-i 1 

< L d* 3 log nE^y[(Ej/jJ IjEj/^a 2 }] 

< Ld [ 2 k (^\^) c W#( J A^)/ 2 log rf/2 n]a -i 5 

where the third inequality follows from (|32|) and the last one from the ele- 
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mentary fact E|X|l||x|> a } < a 1 E|X| 2 . This way we obtain 

\\h n \\j ,J (34) 



< 



S up{E[^ n ,/ )n/i]: Wfih < l,Vjgj 4 || (IEj/f ) 1/2 ||oo < 2 n *^' 2 } 



i=l 



+ 2( k ('^\./) c W~«)#( J AJ)/2 log rf/2 n 
i=o jgji 

i 

< S u V {E[(h n ,fo)Hfi]: Wfih < I^Vjcj, ||(Ej/?) 1/2 ||oo < 2"#WV 2 } 



i=l 



+ L d Y,2 {klin) - n)#IC/2 ^ 



d ' 2 n. 



Let us thus consider arbitrary /j, i = 0, . . . , /c such that ||/j|| 2 < 1, 
|| (E j^/f ) 1/2 ||oo < 2 n# ( Jl \ J )/ 2 for all J C J ; (note that the latter condition 
means in particular that ||/i|U < 2 n # J ^ 2 ). 

We have by assumption (|12|) for sufficiently large n, 

k 

\n(h,f )Hm < \\h\\ K ^ 2nd/ , < L d D\og( d - Ac ^' 2 n. 

i=l 

We have also 

k k 

E l(^/o) 1 {|/ v |2> % 2« d log' i n}II^I - E [l /l l 1 {|fe| 2 >»? d 2™ d log d n}]n H^ll°° 

i=l i=0 

<2^/ 2 E[|/ i |l {|h|2 >, )d2ridlogdn} ] =:a n . 
Also for I C I d , I ^ 0, I d , denoting h n = frl{| ft |2< r?d2 «diog d n}> we get 



E\(h n ,f ) /il{E / H 2 >2"#- rc }l 
j=l 

k 

II(^n/|/,| 2 ) 1/2 ] 



i=0 



< [ J]2"#(^)/ 2]EH(E ^|2 ) i/2 1{E ^ 2n#JC}] 



i=0 



< Lar^^Ej^EjH 2 ^^ 1 / 2 !^^^ 

< L d [2"# /C /2 lo g^/2 n]E/c[(E/F 2 ) l/ 2l{E ^^ 2n#/c}] =; ^ 
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Let us denote h n = h n rb^/c/ d 1 {m I H 2 <2# IC »} and In = E \h n l B iJ 2 . 
Combining the three last inequalities we obtain 



\nK, h) n fi\ ^\ e ^ fo) n f*\ + mhnu^jo) n m 

i=l i=0 i=l 

<L d D log( d - de s ^)/ 2 n + E| {hl m ^ 2nd logd n} J )Y[fi 



i 



/o>n/» 

0^/C/ d i=l 

i 

+^EKM B /,/o)n/ii 

ICI d i=l 

<L d D\og^^n + a n+ £ ^+Y,\R- 

%^I<ZI d ICI d 

Now, combining the above estimate with (j34f) . we obtain 

IKIL^ < ^ E 2^W-^ #/C / 2 log d / 2 n + L^log^ dc ^)/ 2 n (35) 

+ a n + Pn+Yl 
%^I<ZI d ICI d 

Let us notice that 



log* 2 



< oo, 



n 



^ log 2d n 

The first inequality was proved in Step 1. The proof of the second one 
is straightforward. Indeed, we have 

n 6 ^ n 

< L^E/cE/i? 2 = L d EH 2 < oo. 
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The third inequality is implicitly proved in Step 3. Let us however present 
an explicit argument. 

12, - \ 2 



log M n 



l^j 1 {M<??d2" d / 2 log' i / 2 n}^ 

log d ra 



< -L<i^(E/cE/(iT 2 l B /)) 2 < oo 



by Lemma [19] applied to the random variable \/¥,jH 2 . 

We are now in position to finish the proof. Let us notice that we have 
either K(\h\ 2 l^ h ^ <2 2ud^) < 1, or we can use the function 

hli 

g 



-{\\h\ 2 <2 2 " d } 



(E(\h\H m ^ nd} ))^ 
as a test function in the definition of \\h\\j $2 nd i obtaining 

(Ed/i^l^p^w})) 1 / 2 =E(h,g) < \\h\\ IdA2nd < D\og d n 
for large n. Combining this estimate with Corollary [3] we can now write 

J>(| Y, KdK(X? ec )\ > L d (D + C)2" d / 2 log d / 2 n) (37) 



n i|<2™ 

J ci d jev IAJo n 



1 /C2 nd / 2 log d/2 n\2/do g j 



L d \ 2^\\h n \\ Joi j 



+ LdYY exp 



1 / C2 nd ' 2 \og d/2 n \2/(d+#/ c ) 



The second series is convergent by (|33j) . 

Thus it remains to prove the convergence of the first series. By (|35p . we 
have for all Jo, J 



exp 



1 /Clog d/2 n\2/de g> 7 



< ^ exp 



1 



Clog d / 2 



n 



+ exp 
+ exp 



L d V2(k/(n)-n)#/ c /2 l og d / 2 n 

Clog rf / 2 n 



2/degJi 



^VDlog^s^)/ 2 







^2/dcgJ- 


+ exp 



1 /Clog d/2 n\2/degJ 









+ exp 



£<2 v «n 
1 / Clog d / 2 n \2/degJ 
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(under our permanent convention that the values of L d in different equations 
need not be the same). The series determined by the three last components 
at the right-hand side are convergent by (|36p since e~ x < L r x~ r for r > 0. 
The series corresponding to the second component is convergent for C large 
enough and we can take C = L d D. As for the series corresponding to the 
first term, we have, just as in the proof of (|33p for any / C I d , 



(- 



C\og d / 2 



n 



< 



L d \L d 2( k i( n )~ n )# IC / 2 log d/2 n 
1 



2/degJ- 



k n>k,k<£K(I,n) 



exp 



C2 (n-k)#I°/2 



2/degJ 



< K \^ sup exp 

^ n>k,k<£K(I,n) 



2/degJ 



< kJ2e(H 2 1 c i) = KEH 2 < oo. 



We have thus proven the convergence of the series at the left-hand side of 
(j37|) with C < L d D, which ends Step 5. 



Now to finish the proof, we just split T, d for each n into four sets, described 
by steps 1-4 and use the triangle inequality, to show that 



HX.f ec )\>L d D2 nd / 2 log 



d/2 



n 



< oo, 



|i|<2 n 



which proves the sufficiency part of the theorem by Corollary [U 



□ 



8 The undecoupled case 

Theorem 4. For any function h: Y, d — > H and a sequence X\, X2, ■ ■ ■ of 
i.i.d., Yi-valued random variables, the LIL ^ holds if and only if h 

h is completely degenerate for the law of X\ and the growth conditions 
are satisfied. 

More precisely, if ^ holds, then is satisfied with D = L d C and 
conversely, together with complete degeneration and the integrability 

condition imply ^ with C = L d D. 
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Proof. Sufficiency follows from Corollary [6] and Theorem [2j To prove the 
necessity assume that holds and observe that from Lemma [8] and Corol- 
lary h satisfies the randomized decoupled LIL ([8]) and thus, by Theo- 
rem [2J (fTTj) holds and the growth conditions (fT2j) on functions 
are satisfied (note that the || • \\j, u norms of the kernel h(X\, . . . ,Xd) and 
£\ ■ ■ • Edh(Xi, . . . , Xd) are equal). The complete degeneracy of (ip, h) for any 
ip £ H follows from the necessary conditions for real-valued kernels. Since 
by (jlip . Ej/i is well defined in the Bochner sense, we must have Ej/i = 0. □ 
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